Hailing a vehicle

ABSTRACT

In general, an indication is received through a user interface of an intention of a potential rider to use an autonomous vehicle. In response to the receipt of the indication, a hailing request is sent by a signaling mode to at least one autonomous vehicle that can receive the hailing request directly in accordance with the signaling mode.

BACKGROUND

This description relates to hailing a vehicle such as an autonomousvehicle.

As shown in FIG. 1, in a common method for a person 10 to hail a vehiclesuch as a taxicab or a ride-sharing vehicle, the person enters a requestby interacting with a mobile app. The request is processed by the mobileapp and then forwarded through the cellular telephone network or theInternet to a central server 12 where it is processed and then forwardedthrough the cellular telephone network to a driver of the vehicle 14.

SUMMARY

In general, in an aspect, an indication is received through a userinterface of an intention of a potential rider to use an autonomousvehicle. In response to the receipt of the indication, a hailing requestis sent by a signaling mode to one or more autonomous vehicles that canreceive the hailing request directly in accordance with the signalingmode.

Implementations may include one or a combination of two or more of thefollowing features. The user interface is presented through the displayof a mobile device. The indication is received through an input featureof a mobile device. The indication is received through an input featureof a device in a fixed location. The hailing request is sent from amobile device. The hailing request is sent from a device in a fixedlocation. The hailing request is sent by a combination of two or moresignaling modes. The signaling mode includes a visual mode. The visualmode includes a display of graphical elements. The visual mode includesan emitting of light. The signaling mode includes an audible mode. Theaudible mode includes a sound. The audible mode includes an utterance.The hailing request is configured to include hailing information relatedto the intended use of the autonomous vehicle. The hailing informationincludes a destination location. The hailing information includes aclass or type of service. The type of service could be the use by apotential rider. The type of service could be the use for a parceldelivery. The hailing information includes a number of passengers. Thehailing information includes information about the potential rider orparcel to be delivered. The hailing information includes a priority ofservice. A receipt confirmation is received from the autonomous vehicle.In response to receiving the receipt confirmation, a hailingconfirmation is sent to the autonomous vehicle.

In general, in an aspect, a hailing request carried by a signaling modeis received directly at a receiving device of an autonomous vehicle. Thehailing request corresponds to an indication of an intention of thepotential rider to use an autonomous vehicle. The hailing request isprocessed at the autonomous vehicle to determine whether to respond tothe hailing request.

Implementations may include one or a combination of two or more of thefollowing features. The hailing request is received by direct wirelesscommunication. The hailing request is received from a mobile device. Thehailing request is received from a device in a fixed location. Thehailing request is received from or on behalf of the potential rider.The hailing request is received by a combination of two or moresignaling modes. The signaling mode includes a visual mode. The visualmode includes a display of graphical elements. The visual mode includesan image or light. The signaling mode includes an audible mode. Theaudible mode includes a sound. The audible mode includes an utterance.The signaling mode includes a human gesture or facial expression orboth. The hailing request includes hailing information related to theuse of the autonomous vehicle. The hailing information includes adestination location. The hailing information includes a class or typeor priority of service. The hailing information includes a number ofpassengers. The hailing information includes information about thepotential rider or parcel to be delivered. A receipt confirmation issent from the autonomous vehicle. A hailing confirmation is receivedafter the receipt confirmation has been sent.

In general, in an aspect, there is a device for hailing an autonomousvehicle. The device includes an input feature to receive an indicationof an intention of a potential rider to use the autonomous vehicle. Thedevice also includes an output feature to send a hailing request by asignaling mode to at least one autonomous vehicle that can receive thehailing request directly in accordance with the signaling mode.

Implementations may include one or a combination of two or more of thefollowing features. The device includes a mobile device. The device isfixed in a location. The input feature includes a touch-sensitivedisplay. The input feature includes a microphone. The input featureincludes a button. The hailing request carries hailing informationrelated to the use of the autonomous vehicle and the device includes aprocessor to embed at least some of the hailing information in thehailing request. The hailing information includes a destinationlocation. The hailing information includes a class or type or priorityof service. The hailing information includes a number of passengers. Thehailing information includes information about the potential rider orparcel to be delivered. The output feature includes a transmitter. Theoutput feature includes a display. The output feature includes a lightemitter. The output feature includes a sound emitter. The signaling modeincludes a visual mode. The visual mode includes a display of graphicalelements. The visual mode includes an image or light. The signaling modeincludes an audible mode. The audible mode includes a sound. The audiblemode includes an utterance. The signaling mode includes a human gestureor facial expression or both. The hailing request is sent by acombination of two or more signaling modes, The device includes areceiver feature to receive a receipt confirmation from the autonomousvehicle after the hailing request has been sent.

In general, in an aspect, a device for use at an autonomous vehicleincludes a receiving feature to receive a hailing request. The hailingrequest corresponds to an indicated intention of a potential rider touse the autonomous vehicle. The hailing request is received directly inaccordance with a signaling mode from a device in the vicinity of thepotential rider. The device includes a processor to process the hailingrequest.

Implementations may include one or a combination of two or more of thefollowing features. The hailing request carries hailing informationrelated to the use of the autonomous vehicle and the device includes aprocessor to embed at least some of the hailing information in thehailing request. The hailing information includes a destinationlocation. The hailing information includes a class or type of service.The hailing information includes a number of passengers. The hailinginformation includes information about the potential rider or parcel tobe delivered. The receiving feature includes a camera. The receivingfeature includes a light detector. The receiving feature includes amicrophone. The signaling mode includes a visual mode. The visual modeincludes a display of graphical elements. The visual mode includes animage or light. The signaling mode includes an audible mode. The audiblemode includes a sound. The audible mode includes an utterance. Thesignaling mode includes a human gesture or facial expression or both.The hailing request is received by a combination of two or moresignaling modes. The processor includes a process to determine an actionbased on the hailing request. The action includes accepting the hailingrequest. The action includes rejecting the hailing request. The actionincludes providing a pickup location to autonomous driving features ofthe autonomous vehicle. A hailing confirmation is received at the deviceat the autonomous vehicle.

In general, in an aspect, an autonomous vehicle includes a processor andsoftware to be executed by the processor to cause the autonomous vehicleto travel to an identified location. The autonomous vehicle alsoincludes a device (a) to receive a hailing request, the hailing requestcorresponding to an indicated intention of a potential rider to use anautonomous vehicle, the hailing request being received directly inaccordance with a signaling mode from a device in the vicinity of thepotential rider, and (b) to provide a pickup location to the processorbased on the hailing request.

These and other aspects, features, implementations, and advantages canbe expressed as methods, apparatus, systems, components, programproducts, business methods, means or steps for performing functions, andin other ways.

These and other aspects, features, implementations, and advantages willbecome apparent from the following description and from the claims.

DESCRIPTION

FIGS. 1, 2, 9, and 10 are block diagrams.

FIGS. 3, 4, and 8 are schematic views of vehicle hailing.

FIGS. 5, 6, and 7 are display screen shots.

FIG. 11 is a schematic diagram of a vehicle.

FIGS. 12 and 13 are schematic diagrams.

Here we describe a system and techniques for a person to use in hailinga vehicle, such as an autonomous vehicle. As shown in FIG. 2, in someimplementations of the system and techniques that we describe here, auser 16 hails a vehicle 20 by communication that is achieved directlywith the vehicle using one or more wireless signaling modes 18. In somecases, a server 12 can also or alternatively be in direct communicationwith the vehicle.

We sometimes refer to the system and techniques broadly as a “hailingsystem.” We use the term “hailing” or “hail” broadly to include, forexample, any action by any person in any context (for instance apedestrian) to signal, call, flag down, alarm, or in any other way toattract the attention of a vehicle for the purpose of indicating theperson's wish to ride in (alone or with others), hire for service, causea delivery to be made by, or otherwise make use of the vehicle (such asan autonomous vehicle).

Hailing can be done by, through the medium of, or with the assistance ofa hailing device. We use the phrase “hailing device” broadly to include,for example, any apparatus, equipment, instrument, or other deviceassociated with hailing, for example, a phone or other handheld device,a smartwatch or bracelet or other wearable device, or a mounted orotherwise stationary device located on or near a sidewalk or road edge,to name only a few. We use the phrase “potential rider” broadly toinclude, for example, any person in any context who wishes to ride in(alone or with others), hire for service, cause a delivery to be madeby, or otherwise make use of a vehicle (such as an autonomous vehicle).We use the term “hire a vehicle for service” broadly to include, forexample, to obtain access in any way and in any context to a vehicle forthe purpose of, for example, riding in the vehicle, causing a deliveryto be made by the vehicle, or otherwise making use of the vehicle.Although we sometimes refer in our discussion to autonomous vehicles,the hailing system could also be used for semi-autonomous vehicles andin some contexts for other kinds of vehicles driven in other ways. Wesometimes use the term “autonomous vehicle” to refer broadly to any suchvehicle. An autonomous vehicle can be hailed when it is passing along aroadway nearby to a potential rider, or when it is stopped, or in othercontexts.

An autonomous vehicle can drive safely without human intervention duringpart of a journey or an entire journey. As shown in FIGS. 10 and 11, anautonomous vehicle includes sensors, actuators, computers, andcommunication devices to enable automated generation and following ofroutes through the environment. Autonomous vehicles typically containone or more video sensors, at least one of which is typically orientedin the forward direction in order to capture data related to the scenein front of the vehicle. Additionally, autonomous vehicles may containone or more LIDAR sensors. Autonomous vehicles typically are equippedwith a wireless two-way communication capability to access data andinformation stored on servers in a cloud service. As also shown in FIGS.10 and 11, the autonomous vehicle can communicate wirelessly with aserver at a central location.

These and other capabilities can enable the autonomous vehicle to reactto detected visual or other physical features in the environment,including particular shapes, colors, textures, human gestures, and othervisual stimuli. Autonomous vehicles may also be equipped with auditorysensors to capture and react to sounds in the environment, and horns,speakers, bells, or other noisemakers to emit sound. We will refer tothese visual and auditory capabilities and devices and a broad range ofother detection capabilities and devices by the general phrase “stimulusdetectors.”

Thus, as shown in FIG. 12, the stimulus detectors 202 of an autonomousvehicle 200 can include video 204, audio 206, image 208, light 210, RF212, for example.

The autonomous vehicles used with the hailing system that we describehere may also be equipped with lights, display screens, light-emittingdiodes, or other light emitting devices 216 mounted, for example, on theexterior of the vehicle such that they can be viewed from outside thevehicle. Autonomous vehicles may have displays 218, lights, sirens,klaxons, or other sound emitters 220, and other signaling devicesmounted, for example, external to the vehicle. We will refer to thesesignaling devices and techniques and to a broad range of other signalingcapabilities and equipment as “signaling devices.” Often the signalingdevices operate in modes that are perceptible by and understood bypeople.

Conventionally, vehicles (such as taxicabs and vehicles being driven bypeople for ride-hailing services) can be hired for service by apotential rider using a ride hailing software process 230 that may runon a processor unit 232 embedded within a smartphone, laptop, tablet, orother computing device 234. The potential rider 236 typically inputs 238to the ride hailing software process a desired pick-up location and mayalso input a desired drop-off (i.e., a goal) location (e.g., adestination address or street intersection).

In known systems, the desired pick-up location is then communicatedwirelessly to a centralized server tasked with assigning movementdirections to one or more and potentially a large fleet (e.g., hundredsor thousands) of vehicles. The centralized server then runs a routingsoftware process that provides by wireless communication to a particularvehicle the potential rider's pick-up location. The particular vehiclethen is driven to the desired pick-up location to meet the potentialrider. Once the potential rider enters the vehicle and completes anidentification process (not necessarily in that order), the vehicle isconsidered hired for service and begins the journey to the desireddrop-off location.

In some typical use cases of autonomous vehicles, the desired pick-up orgoal location may be specified by an algorithm (which may be located ona centralized server in the cloud and tasked with optimizing thelocations of a fleet of autonomous vehicles with a goal of minimizingrider wait times when hailing a vehicle), or may be provided by anotherprocess (e.g., an emergency process that identifies the nearest hospitalas the goal position due to a detected medical emergency on board thevehicle).

In implementations of the hailing system that we describe here, apotential rider of a vehicle (e.g., an autonomous vehicle) may wish tohire a vehicle for service without, for example, inputting a desiredpick-up location into a smartphone, laptop, tablet, or other computingdevice, but rather to simply hail the vehicle (such as an unoccupiedautonomous vehicle passing on a roadway). The hailing system that wedescribe here can also apply to other scenarios, such as hailing anautonomous vehicle that is parked at a particular location.

As shown in FIG. 3, in implementations of our hailing system, hailing ofan autonomous vehicle 30 can be accomplished by sending information fromor on behalf of or in association with the potential rider 32 to theautonomous vehicle using one or a combination of two or more of avariety of signaling modes including those discussed below. We use theterm “signaling mechanism” broadly to include, for example, any device,equipment, apparatus, or system by which information about the hailingcan be sent to the autonomous vehicle. We use the term “signaling mode”broadly to include, for example, any form in which information can besent by wireless signals or other methods, including non-electronicmethods. Among others, the signaling mode can include displaying animage or sequence of images, emitting a sequence of light signals,emitting a sequence of sound signals, emitting a wireless communicationsignal, or engaging in gestures or facial expressions, to name a few.

In most cases, the information signaling mechanism and mode are designedto increase the incidence rate of true detections (i.e., scenarios inwhich the autonomous vehicle correctly identifies that a hailing requesthas been directed to the vehicle) and to reduce the incidence rate offalse detections (i.e., scenarios in which an autonomous vehicleincorrectly identifies that a hailing request has been directed to thevehicle when none has been sent or one has been sent to another vehicle,for example).

As shown in FIG. 3, the potential rider 32 can signal the autonomousvehicle using a hailing device 34 such as a mobile device (or a widevariety of other devices).

In some instances, the hailing may be performed by a potential rider (orsomeone else, such as a companion) who is carrying a hailing device(e.g., a smartphone). In some cases (see FIG. 4), the hailing may beperformed using a hailing device that is mounted or installed in a fixedlocation, such as on a structure at which vehicle hailing may commonlyoccur (e.g., outside a busy hotel). In some cases, the hailing devicemay be typical human capabilities performed by a potential rider (orsomeone on her behalf) who is not carrying any external hailing device,but rather uses other signaling mechanisms, such as a hand, or facial,or other gesture or whistling, yelling, or making another sound totransmit information. In some implementations, the hailing may beperformed by interacting with a device attached to the autonomousvehicle itself (e.g., a smart card reader that can read the user's smartcard or a smart device that can communicate with the user's smartphone.)

Hailing using the hailing system can be done in a way that complies witha predefined and commonly accepted hailing protocol that could cover themanner in which the signaling mechanism is to operate, the informationthat is to be transmitted, its format, and a wide variety of otheraspects. A wide variety of entities that operate individual autonomousvehicles and fleets of them to be hired for service and entities thatproduce software, hardware, and other equipment for use in autonomousvehicles and hailing devices can subscribe to such a hailing protocol.

A desirable aspect of such a hailing protocol is that it may enable apotential rider to hail a vehicle more quickly (because vehicles of morethan one fleet operator comply with the protocol) than would beachievable by hailing an autonomous vehicle that is operated by onespecific fleet operator under a proprietary hailing method.Interoperability among fleet operators could also require a commoncentral cloud or system that could broadcast the hailing requests to allfleet operators.

Another desirable aspect of the hailing system that we describe here isthat the potential rider may be able to hail a vehicle more quickly thanif the rider were required to input a desired pick-up location (which isnot required in implementations of the hailing system), wait for thesystem to assign a vehicle to her, and then wait for that vehicle toarrive at the pickup location. Another desirable aspect of someimplementations of the hailing system is that the potential rider may beable to hail a vehicle anonymously, e.g., without using a ride hailingsoftware process that may be tied to a user account that includespersonally identifying information. In other words, the potential riderneed not say where she is headed or expose any of her personalinformation in the process of using the hailing system.

On the other hand, depending on the signaling mechanism, certain kindsof information associated with the hailing request can be transmitted tothe autonomous vehicle by the hailing system. This information mayinclude, but not be limited to, information about the number ofpassengers, the desired class or cost of service (which may be used toexclude vehicles of a certain size or class from responding to thehailing request), information about a priority or emergency hailingrequest, or personal identifying information about the rider, or acombination of any two of more of those, and of other information.Transmitted information may employ any of a number of possibleencryption techniques.

The hailing device can include, for example, one or more of thefollowing (and wide variety of others):

1. As shown in the example of FIG. 12, a smart phone, tablet, handheldPC, smartwatch, bracelet, or other wearable device or other configurablemobile device that is equipped with at least a processor 232, a memoryunit 244, an input device or process 246 associated with a userinterface 248, and (in some cases is equipped with) a display screen,light-emitting diode(s), luminescent material, e-ink screen, or otherlight emitting or light modulating media 250. Such display andlight-emitting media may be able to exhibit or modulate sequences ofimages, colors, or lights or a combination of any two or more of themand other visual indications. The hailing device may be equipped withone or more video sensors 262 capable of detecting visual features inthe environment. The device may be equipped with one or more microphonesor speakers (or both) that enable the capability to detect and emitsound 264. The device may be equipped with wired or wireless receiversand transmitters that enable it to communicate with, among other things,a central cloud or servers 242.

2. A printed paper or card.

3. A whistle or other noisemaker, including programmable noisemakersequipped with at least a processor and speaker, and potentially amicrophone.

Modes for Hailing an Autonomous Vehicle

Exemplary methods for hailing an autonomous vehicle include but are notlimited to the following methods, which may be employed eitherindependently or in combination.

Image Display

As shown in FIG. 5, hailing of an autonomous vehicle may be achieved bydisplaying a particular image or sequence of images 44, 46 (for example,displayed alternating at a fixed repeat rate) on a hailing device 48(234 in FIG. 12) in a manner (e.g., presenting the device in anorientation such that the image or images are displayed toward theroadway at shoulder height or above) that is likely to lie within avisual line of sight of, for example, video sensors 204 mounted on apassing autonomous vehicles. For example, the hailing device can belocated at a sidewalk or road edge, or at an intersection where thepotential rider is standing.

The image or images to be displayed may exhibit one or a combination ofany two or more of the following properties:

1. Geometric features 50 (e.g., image features constructed of geometricelements) or patterns of features 52 such as features that are notsmaller than those that can reliably be resolved by an autonomousvehicle at a nominal hailing distance 1-30 meters) using, for example, adetection system based on vision sensor(s) 208 mounted on the vehicleand that have a typical field of view (e.g., 90 degrees horizontal×60degrees vertical) and resolution (e.g., 640 pixels horizontal×480 pixelsvertical). As shown in FIG. 13, the software processes 300 running onthe hailing device can include geometric element processes 302 to formand present the geometric features.

2. Formed in black-and-white (as in FIG. 5), grayscale, or color.Displaying a grayscale or color image rather than a black-and-whiteimage may be advantageous because the grayscale and color image maycontain more information than a black-and-white image and may thereforeenable encoding of a greater amount of information related to thehailing request, or enable reduced incidence of false detections orboth. Detection of color information contained in an image implicitlyrequires that autonomous vehicles be equipped with color video sensors.The software processes 300 running on the hailing device can includeblack-and-white, grayscale, and color processes 304 to produce theappropriate images.

3. Textures (e.g., arrangements of colors or intensities in a region ofan image or the entire image) that can reliably be resolved by anautonomous vehicle at a nominal hailing distance using a detectionsystem based on vision sensor(s) mounted on the vehicle and that have atypical field of view and resolution. The software processes 300 runningon the hailing device can include texture generating processes 306.

4. The texture might include information that is encoded in a format(such as QR codes 60 as shown in FIG. 6, which encode information in abinary format). This allows for more information to be communicated tothe vehicle. Additional information can be encoded in the hailingrequest by increasing the resolution of the format. The softwareprocesses 300 running on the hailing device can include encodingprocesses 308 for this purpose.

5. A sequence of colors 62, 64, 66 can be displayed by the hailingdevice at a fixed repeat rate, as shown in FIG. 7. FIG. 7 shows colouredlight being emitted by a display screen 68 (250 in FIG. 12). Other lightsources, such as LEDs, torches, and flashlights, to name a few, can beused. The emitted light can be in the visible spectrum or the invisiblespectrum. The light could be of a single wavelength or contain multiplewavelengths. The pattern can be changed temporarily as can the intensityof the light. As discussed later, sounds can be used as the signalingmode, and the frequencies of the sound can be controlled similarly.Temporal and spectral control of the light can be handled by lightmodulation processes 310 running on the hailing device 300.

6. The appearance of the image should be uncommon in order to reduce thechance that a similar or identical image that is not produced by ahailing device is detected by sensors on an autonomous vehicle, therebyresulting in a false detection. For example, the image should not besimilar to images that are commonly located on traffic signs or othercommon sidewalk features, such as news boxes, bus stop shelters, etc.

Information related to a hailing request (e.g., the desired class orcost of service, or the presence of a priority or emergency hailingrequest) may be encoded in the image by the encoding and lightmodulation processes 308 and 310 using various methods, including butnot limited to the following individual methods, or combinations ofthese methods:

1. Varying image features, e.g., displaying a unique image or sequenceof images depending on the information related to the hailing request.For example, a particular image may be displayed for a hailing requestby a potential rider who is a single passenger for an autonomous vehicleoffering a standard class of service, while another particular image maybe displayed for a hailing request by a potential rider who representstwo passengers for an autonomous vehicle offering a premium class ofservice.

2. Varying a sequence of images displayed, including varying thetemporal properties of the sequence of images displayed. For example, aparticular sequence of images may be displayed for a hailing request bya potential rider who is a single passenger for an autonomous vehicleoffering a standard class of service, while another particular sequenceof images may be displayed for a hailing request by a potential riderwho represents two passengers for an autonomous vehicle offering apremium class of service. The temporal properties such as the displayduration of each image may be modulated to encode additionalinformation, or reduce incidence of false detections.

3. Varying the black-and-white, grayscale, or color properties of theimage or sequence of images displayed, including varying the temporalproperties of the color of the images. For example, a particular redimage may be displayed for a hailing request by a potential rider who isa single passenger for an autonomous vehicle offering a standard classof service, while the same particular image may be displayed green for ahailing request by a potential rider represents a party of twopassengers for an autonomous vehicle offering a premium class ofservice.

4. Varying the textural properties of an image or sequence of imagesdisplayed. For example, a particular black-and-white checkerboard imagemay be displayed for a hailing request by a potential rider who is asingle passenger for an autonomous vehicle offering a standard class ofservice, while a particular green-and-yellow checkerboard image may bedisplayed for a hailing request by a potential rider who represents aparty of two passengers for an autonomous vehicle offering a premiumclass of service.

5. The image to be displayed may be a company logo or other symbol thatconveys an advertisement or promotional information. The advertisementmay be for an autonomous vehicle company (e.g., a company that operatesan autonomous vehicle that may respond to the hailing request) oranother business entity.

Emitting Light

Hailing of an autonomous vehicle may be achieved by emitting light froma hailing device in a manner (e.g., presenting the device in anorientation such that the emitted light is directed toward the roadwayat shoulder height or above) that is likely to lie within a visual lineof sight of video sensors mounted on an autonomous vehicle (e.g., from asidewalk or road edge, or by a potential rider who is standing at anintersection).

The emitted light to be displayed may exhibit one or a combination oftwo or more of the following properties based on execution of encodingand light modulation processes 308 and 310 on the hailing device 300:

1. The wavelength of the light may lie within the visible spectrum or,given sufficient emission capabilities of the hailing device, the nearinfrared spectrum or other spectrum provided that the wavelength can bedetected by video sensors mounted on the autonomous vehicle.

2. The wavelength of the light in the visible spectrum may lie in bandsassociated with a particular color or many colors.

3. The wavelength of the light may be uniform or variable.

4. The wavelength of the light may be controllable or uncontrollable.

5. The intensity of the light may be uniform or variable. The lightintensity should be reliably detectable by an autonomous vehicle at anominal hailing distance using a detection system based on visionsensor(s) or light detection sensor(s) that have a typical detectionsensitivity.

6. The intensity of the light may be controllable or uncontrollable.

7. The characteristics of the emitted light (e.g., the color,wavelength, intensity temporal frequency (i.e. blinking rate), orcombinations of them) should be uncommon, in order to minimize thechance that emitted light having similar or identical characteristics iserroneously detected by sensors on an autonomous vehicle as a hailingrequest, thereby resulting in a false detection. For example, theemitted light should not be similar in characteristics to the lightemitted by a typical vehicle turn indicator.

Information related to the hailing request (e.g., the desired class orcost of service, or the presence of a priority or emergency hailingrequest) may be encoded in the emitted light by the encoding processes308 using various methods, including but not limited to the followingindividual methods, or combinations of these methods:

1. Varying the light wavelength uniformly on the display of the hailingdevice. For example, the display device may display a color thattransitions from red to green according to a specified temporal pattern.

2. Varying the light wavelength according to specific spatial ortemporal patterns on the display (e.g., distinct spatial areas of thedisplay or the entire display may display distinct light wavelengthsuniformly or in a time varying fashion). For example, a particularcheckerboard image may be displayed with individual cells of thecheckerboard varying in color at distinct frequencies.

3. Varying the light intensity uniformly on the entire display, oraccording to specific spatial or temporal patterns on the display.

Gesturing

As shown in FIG. 8, hailing of a passing autonomous vehicle may beachieved by performing a (e.g., uncommon) gesture or sequence ofgestures 70 at a position and in an orientation that is likely to liewithin visual line of sight of video or LIDAR sensors mounted on anautonomous vehicle (e.g., from a sidewalk or road edge, or when apotential rider is standing at an intersection). A software process 282(FIG. 12) running on a processor 280 mounted on the autonomous vehiclewould then analyze captured data from the video or LIDAR sensors todetect the presence of the gesture(s) that are intended to represent ahailing request.

The gesture(s) to be displayed may exhibit the following properties:

1. The gesture or gestures may comprise static positioning or dynamicmotion of one or more of the arms or legs or both of the potential rideror other person, using positions and motions that are sufficientlydifferent from normal limb positioning and motion to be resolvedcorrectly by an autonomous vehicle at a nominal hailing distance using adetection system based on vision sensor(s) mounted on a vehicle and thathas a typical field of view and resolution. The gesture or gestures mayalso comprise facial expressions which can be resolved correctly by anautonomous vehicle at a nominal hailing distance using a detectionsystem based on vision sensor(s) mounted on a vehicle and that has atypical field of view and resolution. An example of a facial gesture isgazing continuously at a car for some minimum duration of time. Thegesture can be combined with the use of a hailing device 72.

2. The appearance of the gesture or gestures should be uncommon in orderto minimize the chance that a similar or identical gesture that is notbeing performed for the purpose of hailing an autonomous vehicle isdetected by sensors on an autonomous vehicle, thereby resulting in afalse detection. For example, the appearance of the gesture or gesturesshould not be similar to gestures used by cyclists to indicate a turn orby police or flagmen to regulate the flow of traffic.

3. A common gesture (such as waving hands or arms) may be used to hailan autonomous vehicle when combined with a secondary, visuallydistinctive stimulus in order to reduce false detections. One suchsecondary visual stimulus may be the potential rider looking at ortracking the autonomous vehicle by her head pose while executing thecommon gesture. A software process 282 running on a processor 280mounted on the autonomous vehicle would then analyze captured data fromthe video or LIDAR sensors or both to detect the presence of bothgesture and the secondary visual stimulus whose combined occurrence isintended to represent a hailing request.

Information related to the hailing request (e.g., the desired class orcost of service, or the presence of a priority or emergency hailingrequest) may be encoded in the gesture(s) through various methods,including but not limited to the following individual methods, orcombinations of these methods:

1. Varying particular gestures, e.g., displaying a unique gesture orsequence of gestures depending on the information related to the hailingrequest. For example, a particular gesture may be displayed for ahailing request by a single potential rider for an autonomous vehicleoffering a standard class of service, while another particular gesturemay be displayed for a hailing request by a potential rider on behalf ofa party of two passengers for an autonomous vehicle offering a premiumclass of service.

2. Varying the sequence of gestures displayed, including varying thetemporal properties of the sequence of gestures displayed. For example,a particular sequence of gestures may be displayed for a hailing requestby a single potential rider for an autonomous vehicle offering astandard class of service, while another particular sequence of gesturesmay be displayed for a hailing request by a potential rider on behalf ofa party of two passengers for an autonomous vehicle offering a premiumclass of service. The temporal properties such as the display durationof each gesture may be modulated to encode additional information, orreduce incidence of false detections.

Emitting Sounds

Hailing of a passing autonomous vehicle may be achieved by emitting froma hailing device (here including a smartphone or similar device, awhistle or similar device, or the potential rider using her mouth) anuncommon sound or sequence of sounds of sufficient volume that they canbe detected by sensors mounted on a passing autonomous vehicle and notproduce incidences of false detection. The emitting of the sound and thefeatures and characteristics of the emitted sound can be controlled andmodulated by one of the encoding or sound modulation processes 308, 312running on a hailing device.

The emitted sound or sounds to be emitted may exhibit the followingproperties:

1. The sound frequency may lie within the typical human audiblefrequency range or, given sufficient emission capabilities of thehailing device, outside the typical human audible frequency range.Emission of a particular sound frequency depends upon by auditorysensors mounted on the passing autonomous vehicle being able to detectsuch a frequency.

2. The sound frequency in the typical human audible frequency range maylie in bands associated with a particular note or many notes.

3. The sound frequency may be uniform or variable.

4. The sound frequency may be controllable or uncontrollable.

5. The sound volume may be uniform or variable. The volume level shouldbe chosen to be reliably detected by a sound measurement sensor orsensors mounted on an autonomous vehicle with a typical detectionsensitivity and given a typical expected ambient noise level.

6. The sound volume may be controllable or uncontrollable.

7. The sound or sounds can be uncommon, in order to reduce the chancethat a similar or identical ambient sound will be detected by sensors onan autonomous vehicle and interpreted as a hailing request, therebyresulting in a false detection. For example, the sound or sounds shouldnot be similar in nature to common sounds such as car horns, car alarms,or emergency vehicle sirens.

Information related to the hailing request (e.g., the desired class orcost of service, or the presence of a priority or emergency hailingrequest) may be encoded in the emitted sound by the encoding and soundmodulation processes 308, 312 using various methods, including but notlimited to the following individual methods, or combinations of thesemethods:

1. Varying the sound frequency uniformly or according to specifictemporal patterns. The pattern of sound frequencies to be emitted may bea company jingle, song, or catchphrase that conveys an advertisement orpromotional information. The advertisement may be for an autonomousvehicle company (e.g., a company that manages autonomous vehicles thatmay respond to the hailing request) or another business entity.

2. Varying the sound volume uniformly or according to specific temporalpatterns.

Direct Wireless Communication

An autonomous vehicle may be hailed using a hailing device that directlycommunicates wirelessly with the autonomous vehicle without the hailingrequest passing through the central server from the potential rider tothe autonomous vehicle. The hailing device could be a mobile device orcould be fixed at a particular location. The hailing device and acommunication device on the autonomous vehicle both would need tosupport an appropriate communication technology, for example, they wouldboth need to include transmitters, receivers, and communicationsoftware. Many existing communication technologies such as RadioFrequency (RF), Infrared (IR), WiFi, WiMax, and Bluetooth may be used oradapted for this purpose. The hailing device could broadcast the requestto nearby vehicles, and once a particular autonomous vehicle hasresponded it can then communicate exclusively with that vehicle. In thecase of a hailing device, communication process 314 can handle thecommunications and compliance with the appropriate protocols.

Direct wireless communication can achieved, in some examples, by use ofvehicle to infrastructure (V2I) compatible protocols. In such examples,a hailing device can send a hailing request to infrastructure (using theV2I compatible protocol). The infrastructure then relays the hailingrequest to vehicles in the vicinity (say, within a range of 500 meters),Autonomous vehicles pick up the relayed hailing requests using theirV2I-compatible receivers. This mode of hailing is direct in that nocentral server is involved. The infrastructure could be a wide range ofdevices such as WiFi access points mounted on, say, streetlights.

Combinations of Hailing Modes

Each of the various signaling modes that we have discussed can be usedin combination with one or more of the other modes, and with additionalmodes not mentioned explicitly. Using modes in combination can reducethe incidence rate of false detections. For example, a hailing methodmay rely on executing a particular gesture (e.g., waving one's arm aboveone's head) while displaying a hailing device that is emitting aparticular temporal sequence of colors. A wide range of othercombinations of signaling modes would also be possible.

Fixed Hailing Device

As mentioned above, an autonomous vehicle may also be hailed by ahailing device, such as a hailing device forming part of a kiosk that isfixed at a location. This is particularly relevant, but not limited, toareas where autonomous vehicles are commonly hailed, or where there isusually a good chance of an autonomous vehicle being present nearby, orwhere there is high pedestrian throughput, or at autonomous vehiclestations. A hailing device in a fixed location could provide a userinterface in a variety of modes, from a simple button to something morecomplex that includes a screen, speakers, microphone, touch-screen, or acombination of these and similar input/output components. The userinterface could enable the potential rider (or someone on his behalf) toindicate a destination location, a number of riders, a class of service,a time to arrive, and a variety of other pieces of information relevantto the hailing. The hailing device at the fixed location could broadcastthe hailing request to one or more nearby vehicles either by attemptingto communicate directly with the one or more nearby vehicles or bycommunication through a central system, using appropriate wirelesscommunication devices including receivers and transmitters for thispurpose.

Processing Received Hailing Requests

Once a hailing request receiver on the autonomous vehicle (e.g. one ofthe communication devices 240 shown in FIG. 12) detects a hailingrequest communicated using one of the signaling modes and devicesdescribed above or other similar signaling modes or devices, a processoron the autonomous vehicle processes the information contained in thehailing request and, among other things, decides whether to accept therequest. This decision may be made by the process running on theautonomous vehicle or by a central system to which the vehicle isconnected or by a combination of the two. The decision can be based on acombination of a number of factors. These factors include, but are notlimited to: the vehicle's current location, the vehicle's current speed,current traffic conditions in the surrounding area, and the vehicle'sdetermination of its ability to come to a safe stop and pick up thepassenger. Nearby autonomous vehicles may also communicate with eachother using vehicle-to-vehicle (V2V) communication to ensure thatmultiple vehicles do not respond to the same hailing request.Communication between vehicles between for this purpose could also beachieved using a centralized system.

Communication Between an Autonomous Vehicle and a Hailing Device

Although we have described various methods for hailing an autonomousvehicle by sending a hailing request from (or on behalf of) thepotential rider or from a hailing device, thereby yielding one-waycommunication between the potential rider and the autonomous vehicle, insome cases it may be desirable to enable two-way communication betweenthe potential rider (or an associated hailing device) and the autonomousvehicle, for the purpose of confirming a hailing request.

To confirm a hailing request, upon detection of, for example, animage-based, light-based, auditory, gestural, or other hailing requestfrom a potential rider, the autonomous vehicle may transmit animage-based, light-based, or auditory receipt confirmation signal, forexample, with the intent that this receipt confirmation would beuniquely identifiable as a response signal by a sensor or sensorsmounted on the hailing device. Upon receipt of this receipt confirmationby the hailing device, the hailing device may transmit a hailingconfirmation signal to the autonomous vehicle that is different from theoriginal hailing request. Upon receipt of this hailing confirmation bythe autonomous vehicle, the autonomous vehicle would initiate a pickupsequence including stopping near the potential rider for the purpose ofpicking up the rider.

Exemplary signaling modes for sending a receipt confirmation includethose for sending a hailing request. More specifically, such methodsinclude but are not limited to:

1. Responding to a hailing request by display of a black-and-white,grayscale, or color image or sequence of images on a display screenmounted on (for example on the outside of) the autonomous vehicle. Theimage properties (e.g., geometric features, texture, and appearance)should be chosen such that they can reliably be resolved by a visionsensor or sensors mounted on the hailing device and having a typicalfield of view and resolution.

2. Responding to a hailing request by emitting light or sound from oneor more display screen, light emitting device, light-emitting diode,siren, klaxon, or other signaling device mounted on (for example, on theexterior) of the autonomous vehicle. The light intensity should bechosen such that it can reliably be detected by a vision sensor orsensors or light detection sensor or auditory sensors mounted on thehailing device and having a typical detection sensitivity, or perceivedby a potential rider.

3. Responding to a hailing request by emitting from one or more speakersmounted on (for example, on the exterior of) the vehicle a sound orsequence of sounds. The volume level should be chosen such that it canreliably be detected by a sound measurement and sensor or sensorsmounted on a hailing device and having a typical detection sensitivity,or perceived by a potential rider, given a typical expected ambientnoise level.

The image or images displayed, light or lights displayed, and sound orsounds emitted by the autonomous vehicle may exhibit exemplaryproperties that include, for example, those described above for image orimages displayed, light or lights displayed, and sound or sounds emittedby the hailing device. Information may also be encoded in the image orimages displayed, light or lights displayed, and sound or sounds emittedby the autonomous vehicle using, for example, the methods describedabove for the image or images displayed, light or lights displayed, andsound or sounds emitted by the hailing device.

Once a receipt confirmation has been received by the hailing device, thehailing device may transmit a hailing confirmation to the autonomousvehicle. Signaling modes for sending for the hailing confirmation caninclude those that can be employed for transmitting a hailing request.However, it may not be necessary to encode within the hailingconfirmation signal information related to the hailing request (e.g.,the desired class or cost of service, or the presence of a priority oremergency hailing request). As a result, the hailing confirmation signalcould be simpler than the hailing request.

Communication may also be verbal through speech output (utterances) byone or more speakers mounted on the exterior of the vehicle and verbalresponses (utterances) received from the potential rider through one ormore microphones mounted on the autonomous vehicle. In order to do so, aprocessor on the autonomous vehicle may execute a speech synthesisprogram or dialog system contained on the vehicle, play back recordedspeech, or broadcast speech received from a human remote operatorconnected to the vehicle via a wireless link. The volume may be chosento be appropriate for the distance of the potential rider to theautonomous vehicle. Once a verbal response has been received by theautonomous vehicle, a speech recognizer or decoding program on thevehicle may determine whether to interpret the received voice signal asa hailing confirmation of the hailing request.

As shown in FIG. 9, in some examples of the hailing system, the processof hailing an autonomous vehicle, begins with the potential ridercausing a hailing request to be sent 90 using a signaling mode offeredby a hailing device or by gesturing, for example. The autonomous vehicledetects the hailing request and decodes the information embedded in it92. A process running on the autonomous vehicle decides to accept orreject the request 94 and in doing so may communicate with a centralserver or other vehicles. The autonomous vehicle then sends a responseto the hailing device 96 for the user using a signaling mode. Thehailing device may forward the response to the user 98 by giving theuser notice on a mobile device or a kiosk using sounds or visualindicators. If the autonomous vehicle accepts the request, it changesits path accordingly and stops safely near the rider 100. In someimplementations, not all of the steps shown in FIG. 9 need be performed.

Other implementations are within the scope of the following claims.

The invention claimed is:
 1. A method comprising: receiving, at an imagecapture device of an autonomous vehicle (AV), one or more pulses oflight emitted by a display of a device at a geographic location;decoding, by one or more processors of the AV, the one or more pulses oflight; identifying, by the one or more processors and based on the oneor more decoded pulses of light, a request for transportation servicesat the geographic location; selecting, by the one or more processors, astopping place at the geographic location; and causing, by the one ormore processors, the AV to stop at the selected stopping place.
 2. Themethod of claim 1, further comprising: in response to identifying therequest for transportation services, sending, by the AV, a receiptconfirmation to a user device.
 3. The method of claim 1, wherein thepulses of light are in a near infrared spectrum.
 4. The method of claim1, wherein the request for transportation services includes adestination location.
 5. The method of claim 1, wherein the request fortransportation services includes a number of riders.
 6. A methodcomprising: receiving, at an audio subsystem of an autonomous vehicle(AV), one or more audio signals emitted from a user device at ageographic location; decoding, by one or more processors of the AV, theone or more audio signals; identifying, by the one or more processorsand based on the one or more decoded audio signals, a request fortransportation services at the geographic location; selecting, by theone or more processors, a stopping place at the geographic location; andcausing, by the one or more processors, the AV to stop at the selectedstopping place.
 7. The method of claim 6, further comprising: inresponse to identifying the request for transportation services,sending, by the AV, a receipt confirmation to the user device.
 8. Themethod of claim 6, wherein the request for transportation servicesincludes a destination location.
 9. The method of claim 6, wherein therequest for transportation services includes a number of riders.
 10. Oneor more non-transitory computer storage media encoded with instructionsthat, when executed by one or more computers, cause the one or morecomputers to perform operations comprising: receiving, at an imagecapture device of an autonomous vehicle (AV), one or more pulses oflight emitted by a display of a device at a geographic location;decoding, by one or more processors of the AV, the one or more pulses oflight; identifying, by the one or more processors and based on the oneor more decoded pulses of light, a request for transportation servicesat the geographic location; selecting, by the one or more processors, astopping place at the geographic location; and causing, by the one ormore processors, the AV to stop at the selected stopping place.
 11. Oneor more non-transitory computer storage media encoded with instructionsthat, when executed by one or more computers, cause the one or morecomputers to perform operations comprising: receiving, at an audiosubsystem of an autonomous vehicle (AV), one or more audio signalsemitted from a user device at a geographic location; decoding, by one ormore processors of the AV, the one or more audio signals; identifying,by the one or more processors and based on the one or more decoded audiosignals, a request for transportation services at the geographiclocation; selecting, by the one or more processors, a stopping place atthe geographic location; and causing, by the one or more processors, theAV to stop at the selected stopping place.
 12. An apparatus comprising:a display screen; one or more processors; memory storing instructionsthat when executed by the one or more processors, cause the one or moreprocessors to perform operations comprising: receiving, by the one ormore processors, user input requesting transportation services;modulating light emitted from the display screen, the modulated lightencoding the request for transportation services; and receiving, by theone or more processors, a message from a vehicle acknowledging therequest for transportations services.
 13. An apparatus comprising: anaudio subsystem including one or more microphones and one or moreloudspeakers; one or more processors; memory storing instructions thatwhen executed by the one or more processors, cause the one or moreprocessors to perform operations comprising: receiving, by the one ormore microphones, speech input; determining, by the one or moreprocessors, that the speech input is requesting transportation services;modulating audio emitted from the loudspeaker, the modulated audioencoding the request for transportation services; and receiving, by theone or more processors, a message from a vehicle acknowledging therequest for transportations services.